Skip to content

test(webapp): chat.agent durability regression suite - #4550

Open
kathiekiwi wants to merge 16 commits into
feat/query-safety-tri-11165from
test/chat-agent-durability-tri-11166
Open

test(webapp): chat.agent durability regression suite#4550
kathiekiwi wants to merge 16 commits into
feat/query-safety-tri-11165from
test/chat-agent-durability-tri-11166

Conversation

@kathiekiwi

@kathiekiwi kathiekiwi commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

What & why

Test-only hardening for chat.agent durability. chat.agent gets its durability from the primitive — object-store snapshot + S2 .in/.out replay + continuation boot — but several store-level mechanisms that replay lands on had no regression test, and two of them were criticals in the 2026-06-10 chat.agent audit: cross-tenant isolation and no duplicate mid-stream turn. This adds those cases against a real Postgres table (testcontainers, no mocks). TRI-11166.

Stack

Stacked on #4549 (query boundary). Merge that first.

What's inside

# Mechanism Test Coverage Control-broken
1 Cross-tenant isolation (audit critical) dashboardAgentTenantIsolation.test.ts Full at the store seam we own: getChatMessages / getSession / chatExists / listChats / countUserMessages / appendChatMessageOnce all refuse a foreign (org, user) — a foreign tenant reads not-found, never a transcript or the session's public access token Yes
2 No duplicate mid-stream turn (audit critical) dashboardAgentDurableResume.test.ts Full: a streamed-then-resumed turn finalises in place and appends nothing; row count and the position allocator both pinned Yes
3 Crash-resume reconstructs state dashboardAgentDurableResume.test.ts Full: replay keeps the mid-turn append, finalises the turn's own message, loses no messages, and rebuilds the session cursor read back via getSession
4 Mid-stream refresh resumes in-flight turn (Last-Event-ID) dashboardAgentDurableResume.test.ts Seam-only: the cursor getSession hands a refreshed client, and a later turn advances (never appends) it. Client-side reconnect / Last-Event-ID replay is already covered in packages/trigger-sdk/src/v3/chat.test.ts — not duplicated
5 Snapshot write-failure path dashboardAgentDurableResume.test.ts Full at this seam: a persistTurn that throws commits nothing (no rows, allocator untouched, cursor unchanged — the version-mismatch case), and the retry replays with no loss
6 .out trimming / OOM retry restarts cleanly dashboardAgentDurableResume.test.ts Seam-only: a restarted turn that re-sends its snapshot loses no data and doubles nothing. .out trimming and the OOM restart itself are inside the closed primitive (not reachable)

The "Full" vs "Seam-only" column is the honest distinction: full means the whole mechanism is exercised from the repos we own; seam-only means we pin the store contract the primitive depends on, and the primitive-internal half lives where we can't reach it.

Key decisions

  • The two criticals were control-broken first. Isolation: removing the organizationId filter from getChatMessages leaks org A's transcript to the owner's user id under another org — the test fails at toBeNull(). No-duplicate: removing the stored-id drop in storeChatMessages makes a replayed persistTurn over-reserve positions (next free slot jumps 3 → 7) — the test fails on the allocator assertion. Both reverted.
  • Real Postgres, no mocks. testcontainers against an actual table, so the store contract is proven, not asserted against a stub.

Residual follow-ups

These live inside the closed chat.agent primitive package and can't be unit-tested from the repos we own; the tests above are the store-level backstop they depend on:

  • The snapshot URL's own auth gate (the audit's snapshot-URL auth gap) — enforced in the primitive; here we prove the webapp store never hands a foreign tenant the PAT it would boot from.
  • Object-store snapshot write + S2 .in/.out replay at the transport level.
  • .out trimming never dropping in-flight data, and the OOM restart mechanism itself.

Testing

pnpm run test --filter webappdashboardAgentTenantIsolation.test.ts and dashboardAgentDurableResume.test.ts.

A chat/session belongs to one (org, user) pair. Pin that every store read — getChatMessages, getSession, chatExists, listChats, countUserMessages — and appendChatMessageOnce refuse a foreign tenant, so a chatId from another org reads as not-found and never leaks a transcript or the session's public access token. TRI-11166.
The primitive resumes a turn by replaying its snapshot; pin the store seam that replay lands on: a streamed-then-resumed turn is not double-appended, a crash mid-turn keeps the mid-turn append and rebuilds the session cursor, a failed write commits nothing and the retry replays with no loss, and an OOM restart replays idempotently. TRI-11166.
@changeset-bot

changeset-bot Bot commented Aug 10, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: bc3e5d0

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 26428fe9-f41a-4b27-b078-1e7ea60dbe9e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pkg-pr-new

pkg-pr-new Bot commented Aug 10, 2026

Copy link
Copy Markdown

Open in StackBlitz

@trigger.dev/build

npm i https://pkg.pr.new/@trigger.dev/build@7428874

trigger.dev

npm i https://pkg.pr.new/trigger.dev@7428874

@trigger.dev/core

npm i https://pkg.pr.new/@trigger.dev/core@7428874

@trigger.dev/python

npm i https://pkg.pr.new/@trigger.dev/python@7428874

@trigger.dev/react-hooks

npm i https://pkg.pr.new/@trigger.dev/react-hooks@7428874

@trigger.dev/redis-worker

npm i https://pkg.pr.new/@trigger.dev/redis-worker@7428874

@trigger.dev/rsc

npm i https://pkg.pr.new/@trigger.dev/rsc@7428874

@trigger.dev/schema-to-json

npm i https://pkg.pr.new/@trigger.dev/schema-to-json@7428874

@trigger.dev/sdk

npm i https://pkg.pr.new/@trigger.dev/sdk@7428874

commit: 7428874

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants